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Amac: Depresyon tanisi olan ¢calisanlar icin prognostic model gelistirmeyi 
amacladik. Gereg ve Yéntem: SSQ Life insurance Company Inc. datasetini 
kullanarak prognostic model gelistirdik. Toplam 1,113 katilimci dahil edildi. 
Bulgular: Quebec eyaletinde yasayanlarda ise dénme siiresi daha kisa olarak 
tesbit edildi. Tartisma: Prognostik modelin kalibrasyonu ve diskriminasyonu 


yeterli olarak tesbit edildi. 
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Abstract 

Aim: Major Depressive Disorder is associated with decreased productivity, 
and is a common cause of disability. We used administrative data from a 
private Canadian disability insurer, SSQ Life Insurance Company Inc. (SSQ), 
to develop a prognostic model for disability resolution. Material and Method: 
We used parametric survival models (Royston-Parmar models) that allow 
more flexibility for the shape of the survival distribution. We employed a 
random split technique with a ratio of 70:30 for creating a derivation and 
validation set. Results: Our study cohort was comprised of 3,113 long term 
disability (LTD) claimants with a primary diagnosis of Major Depressive Dis- 
order, of which 2,679 cases (87%) were closed when our data was captured 
and 434 (13%) were censored. The median time on LTD benefits was sig- 
nificantly shorter among disabled workers residing in Quebec (approximately 
200 days) than in Ontario (approximately 550 days, p<0.001). Our final fitted 
model demonstrated that older age, female gender, shorter claim approval 
time and residing in Ontario vs. Quebec were associated with slower LTD 
claim closure among workers disabled due to depression. We produced a 
receiver operating characteristic curves for 1-year follow-up period for the 
training and validation datasets and area under the curve estimates were 
0.70 and 0.69, respectively. Discussion: Our prognostic model has modest 
but acceptable discrimination and good calibration. We confirmed that work- 
ers receiving LTD benefits for depression who reside in Quebec, Canada are 
more likely to resolve their insurance claims quickly than those who reside 
in Ontario, Canada. 
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Introduction 

Major Depressive Disorder is characterized by depressed mood; 
suicidal thoughts; and diminished sleep, energy, appetite and 
physical activity for at least two weeks [1]. Worldwide, the point 
prevalence of depression is 5-9% in women and 2-3% in men 
and the lifetime prevalence rate is 16.6% [2, 3]. 

The presence of depression has been linked to negative health 
outcomes, including cardiovascular disease, diabetes mellitus 
and increased risk of mortality [4]. Depression also has been 
associated with productivity loss and disability [5]; individuals 
with depression are seven-fold more likely to develop disability 
than the general population [6]. 

We recently explored factors associated with long-term disabil- 
ity (LTD) claim duration using administrative data from a large, 
private Canadian insurer (Sun Life Canada) [7]. Our adjusted 
analysis found an unexpected association between province of 
residence and claim duration; specifically, LTD claimants resid- 
ing in Quebec were more likely to resolve their claim faster than 
similar patients residing in Ontario who had depression (hazard 
ratio [HR] = 1.93; 99% confidence interval [Cl] = 1.78 to 2.09) 
[7]. 

In this study, we used the administrative data of another Cana- 
dian disability insurer— SSQ Life Insurance Company Inc. (SSQ) 
to confirm our previous research findings of variations in claim 
closures in disability claimants with depression. The aim of this 
study is to develop and validate a prognostic model for predict- 
ing the possibility of long-term disability (LTD) claim closure in 
workers disabled by depression. 


Material and Method 

Standardized reporting 

We followed the recommendations from the reporting state- 
ment and checklist-—Reporting Recommendations for Tumour 
Marker Prognostic Studies (REMARK) and quality assessment 
tool—Quality in Prognosis Studies (QUIPS) [8]. 


Patients 

We undertook a prognostic model study using a retrospective 
cohort design. We examined all claims that SSQ approved for 
LTD benefits from January 1, 2007 to March 31, 2014. Our study 
sample consisted of 22,205 LTD claims. We excluded claimants 
who received a General Expenses benefit plan, as well as those 
for whom the case managers did not record an initial decision 
to maintain data integrity (Figure 1). 

LTD benefits provide wage replacement benefits for disabled 
workers up to age 65, typically as long as they remain disabled 
from their own occupation for the initial two years of the claim, 
and disabled from any and all occupations for which they are 
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Figure 1. Flow diagram of long-term disability claimants. 
Note: LTD indicates long-term disability claimants 


qualified by training or experience after they have been on claim 
for two years (the change of definition period). For our analy- 
ses we included all unique claims that were approved for LTD 
benefits with a primary diagnosis of Major Depressive Disorder 
(henceforth referred to as depression). We excluded all individu- 
als whose claims were recorded as closed prior to contractual 
approval. The study was approved by the Research Ethics Board 
at McMaster University. 

SSQ administrative database consists of demographic, admin- 
istrative, and clinical information. All data is entered by the 
case manager(s) overseeing each claim. We screened all data to 
identify outliers, inconsistencies and missing data by calculat- 
ing summary statistics and exploring distributions graphically. 
If clear outliers and inconsistencies were identified, we worked 
with SSQ to correct the data. 

The principal prognostic variable was claimant’s province of 
residence (Ontario or Quebec). We also included age, gender, 
pre-disability salary, physical workplace demands and time to 
claim approval in our statistical model as candidate prognos- 
tic markers. Our primary outcome was time to LTD claim clo- 
sure, defined as the duration from claim approval until claim 
closure. We performed a time-to-event analysis using flexible 
parametric models to assess the association between time to 
claim closure and prognostic markers [9, 10]. For LTD claims 
that were unresolved when the data was extracted, we used 
the date of extraction as our censoring point. We defined, a 
priori, the following five prognostic markers based on previous 
research, consultation with experts in the field, and availability 
in the administrative database: (1) province of residence, (2) 
age, (3) gender, (4) pre-disability salary, (5) physical workplace 
demands and (6) time to claim approval (Table 1) [7]. 


Table 1. Description of variables and anticipated direction of the effects for 
long-term disability closure 


Predictor Description Anticipated direction 
of claim closure 
Age Age of claimant at Older age: (-) 
disability 
Gender Gender of claimant Female: (-) 
Province of residence Province the claimant Ontario: (-) 


resided in 


Pre-disability salary Salary of claimant Higher salary: (-) 


Physical job demand Physical demands on Higher physical 


demands claimant's job demands: (-) 

Duration of claim Longer time to claim Longer time (-) 
approval approval 

Note: (-) = associated with slower claim closure; (+)= associated with faster 


claim closure. 


Statistical analysis 

We present baseline demographics as frequency (percentage) 
for categorical variables and means and standard deviations 
for continuous variables. We tested the differences between 
LTD claimants who resided in Ontario and Quebec with the 
Pearson’s Chi Square test for categorical variables and two- 
sample t-tests for continuous variables. 

Pair-wise correlations between predictor variables using the 
Pearson correlation coefficient (r) were tested. We planned to 
remove the variable with lesser importance if r is greater than 
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0.80 for 2 predictors. The linear regression model was used to 
generate collinearity statistics. Tolerance and variance inflation 
factor (VIF) were used to test the assumption. Values less than 
10 for VIF, and more than 0.1 for tolerance were considered 
violations. All the values were below the limits for r and VIF. 
We used spline-based survival models (Royston-Parmar) which 
allow more flexibility for the shape of the survival distribution 
and employ restricted cubic spline (RCS) of log-time. We started 
fitting a model by incorporating candidate prognostic mark- 
ers—province of residence, age, gender, pre-disability salary, 
time to claim approval, and physical workplace demands—into 
the models for estimating survival. For variable selection, we 
employed psO.1 as the significance threshold to be retained in 
the model with a backward elimination approach [11]. We em- 
ployed parsimonious prediction models by retaining the most 
significant factors for our final model [12]. We used a log-like- 
lihood ratio test, Akaike Information criterion (AIC) and Bayes- 
ian Information criterion (BIC) to guide decisions regarding 
the number of optimal nodes and best fitted model in terms of 
variable selection and time-dependency of variables [13]. A risk 
factor can be fixed (i.e., race) or change over the follow-up time 
(i.e., weight) [14] and may need to be included as time-varying 
covariate. Province of residence interacted with time and we 
used spline function to address time-dependency with province 
of residence. 

We fitted our flexible parametric models using freely available 
stpm2 software developed by Lambert and Royston for the 
Stata package. We estimated descriptive summary measures 
using Stata/SE 12.1 (Stata Corporation, College Station, TX). 
We employed a random split technique with a ratio of 70:30 
for creating derivation and validation datasets. We used the 
baseline hazard function and f-coefficients from the best fit- 
ted model to calculate the probability for claim closure in the 
validation set. Calibration indicates the amount of agreement 
between the predicted and observed risks. Calibration curves 
indicate expected versus predicted probability of LTD claim clo- 
sure based on deciles of risk. LTD claimants are placed in prob- 
ability quantiles ranked on the horizontal axis and vertical axis 
from lowest to highest. Predicted probability quantiles were 
plotted on the horizontal axis while observed probability quan- 
tiles are plotted on the vertical axis. We visually examined the 
calibration plots and employed the Hosmer-Lemeshow X2 test 
to examine the agreement between the observed and predicted 
probabilities. P-values above 0.05 indicated adequate calibra- 
tion for the models. 

Discrimination implies the ability to predict those with and 
without the outcome of interest. We assessed discrimination 
using a receiver operating curve (ROC) which plots sensitivity 
against 1 minus specificity on different cut-off values [15]. Our 
reference standard was observed disability claim closure from 
the SSQ database. Area under the curve (AUC) between 0.90 
to 1.0 is considered excellent, whereas values between 0.80 to 
0.90 are good, 0.70 to 0.80 are fair, 0.60 to 0.70 are poor, and 
less than <0.60 are fail [24]. 

We employed two-sided tests with a significance level of <0.05. 
All data analyses were performed using Stata (StataCorp. 
2013. Stata Statistical Software: Release 13. College Station, 
TX: StataCorp LP). 


Results 

The descriptions of variables and anticipated direction of the 
effects for long-term disability closure are provided in Table 1. 
Our study cohort was comprised of 3,113 LTD claimants with 
depression out of which 2,679 cases (87%) were closed when our 
data was captured, and 434 (13%) were censored. A total of 198 
claimants resided in Ontario (mean age 45 years) while 2,915 
LTD claimants resided in Quebec (mean age 44 years) (Table 2). 
Duration of claim approval was longer and salary was higher in 
Ontario (Table 2). Flow diagram of long-term disability claimants 
is depicted in Figure 1. 

Figure 2 presents Kaplan-Meier Curves for the proportion of LTD 
claimants remaining on benefits, stratified by province of resi- 
dence. The median time on LTD was approximately 200 days in 
Quebec and 550 days in Ontario. The log-rank test for equality 
of survivor functions between provinces indicated that the curves 
were significantly different (p=0.0001). A graphical display shows 
hazard rates of long-term disability claimants remaining on ben- 
efit in Ontario and Quebec (Figure 3). The difference in hazard 
rate is higher at 500 days (Figure 3). 

The final fitted model included age, gender, salary, physical work- 
place demands, time to claim approval and province of residence. 
Based on AIC and BIC criteria, the optimal number of knots was 
five under the hazard scale in our models (Table 3). Our Royston- 


Table 2. Characteristics of long-term disability claimants’ with major depres- 
sion 


Variable Ontario_LTD QuebecLTD P-value 
Claims Claims 

Total claims 198 2915 

Age of claimant, mean (SD), yr 45 (9) 44 (9.3) 0.10 

Sex 

Female, n (%) 160 (80%) 1828 (62%) 0.0001 

Male, n (%) 38 (20%) 1087 (38%) 

Monthly salary, mean (SD), $ 4,556 (3647) 3,952 (1639) 

Physical demands 0.0001 

Sedentary, n (%) 95 (48%) 1427 (48%) 

Light, n (%) 94 (48%) 991 (34%) 0.0001 

Heavy, n (%) 9 (4%) 497 (18%) 

Duration of claim approval, 37 (24) 27 (22) 0.0001 


mean (SD), days 


Note: SD indicates standard deviation. 


Kaplan-Meier survival estimates 
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Figure 2. A graphical display that shows proportion of long-term disability claim- 
ants remaining on benefit in Ontario and Quebec. 
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Figure 3. A graphical display that shows hazard rates of long-term disability 
claimants remaining on benefit in Ontario and Quebec 


Table 3. Criteria for deciding on number of knots in the Royston-Parmar models 
using the proportional hazard family 

Number of knots (df) AIC BIC 

1 (df=2) 7061 7135 

2 (df=3) 7063 7143 

3 (df=4) 7064 7149 

4 (df=5) 7066 7157 

5 (df=6) 7053 7150 

6 (df=7) 7047 7155 


Note: df= degrees of freedom (1-number of knots); AIC indicates Akaike Infor- 
mation criterion; BIC indicates Bayesian Information criterion. We choose five 
knots at minimum BIC at baseline. 


Parmar models for LTD claimants indicated older age (HR=0.81, 
99% Cl= 0.77-0.85, per decade), female sex (HR=0.82, 99% Cl= 
0.74-0.91) and shorter time to claim approval (HR=0.98, 99% Cl= 
0.97-0.99, per day) were associated with slower LTD claim clo- 
sure. Furthermore, LTD claimants with depression who resided 
in Quebec were more likely to have shorter disability closure as 
compared to Ontarians (HR=2.3, 99% Cl= 1.73-3.2) (Table 4, Fig- 
ure 3). 

We assessed prognostic model performance. The 3,113 claim- 
ants with a major depression diagnosis were randomly clas- 
sified as a derivation dataset (2/3, 70/100, n=2179) and a 
validation dataset (1/3, 30/100, n=934). There was not a sig- 
nificant difference between the derivation and validation data- 
sets; therefore, we did not employ stratification in order to bal- 
ance prognostic markers between the groups (Table 5). Figure 
4 presents the observed versus predicted 1-year rate of LTD 
claim closure in the derivation dataset, demonstrating rela- 
tively small differences between predicted and observed values 
and no systematic over or underestimate. We produced an ROC 
curve for 1-year follow-up period in the training and validation 
datasets. The area under the curve estimates were 0.70 for the 
derivation dataset and 0.69, for the validation dataset (Figures 
5 and 7). Figure 6 depicts the calibration plot of the validation 
dataset which did not show a significant difference between 
expected versus predicted probability of long-term disability 
claim closure based on deciles of risk (P > 0.05). 


Table 4. Determining factors predictive for time to long-term disability benefit 
closure based on parametric survival models (Royston-Parmar models) 


Factor HR (99% Cl) P-value 
Age (per decade) 0.81 (0.77 to 0.85) 0.0001 
Sex 

Female vs male (reference) 0.82 (0.74 to 0.91) 0.0001 
Pre-disability salary (per month): 0.99 (0.99 to 1) 0.46 
Job demands 

Heavy vs. sedentary (reference) 1.05 (0.94 to 1.16) 0.34 
Light vs. sedentary (reference) 0.98 (0.86 to 1.12) 0.83 
Province 

Quebec vs. Ontario by major depression 2.3 (1.73 to 3.2) 0.0001 
Duration of claim approval (days) 0.98 (0.97 to 0.99) 0.0001 


Note: Cl= confidence interval; HR= hazard ratio; HR <1 reflects a longer time to 
claim closure; HR >1 reflects a shorter time to rclaim closure. 


Table 5. Characteristics of long-term disability claimants’ in the derivation and 
validation datasets 


Variable Derivation Validation P-value 
dataset dataset 
(n=2179) (n=934) 
Age of claimant, mean (SD), yr 44 (9.3) 44 (9.2) 0.21 
Female gender, n (%) 1392 (63%) 596 (63.8%) >0.05 
Monthly salary, mean (SD), $ 4012 (1921) 3938(1628) 0.30 
Physical demands >0.05 
Sedentary, n (%) 1069 (49%) 453 (48%) 
Light, n (%) 751 (34%) 334 (35%) 
Heavy, n (%) 359 (16%) 147 (15%) 
Duration of claim approval, mean 27 (22) 28 (22) 0.49 


(SD), days 


SD indicates standard deviation. 
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Figure 4. Expected versus predicted probability of long-term disability claim clo- 
sure based on deciles of risk is depicted in the derivation dataset. The corre- 
sponding Hosmer-Lemeshow test was 10.7 with a p-value of 0.1. 


Discussion 

Major Findings 

We confirmed that LTD claimants with depression who resided 
in Ontario remained on LTD disability benefits longer than those 
who resided in Quebec. Our study also confirmed that older age, 
female gender, and longer time to claim approval were associ- 
ated with longer LTD claim duration. Application of the model 
showed modest but acceptable calibration at 1 year (0.70) and 
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Figure 5. Receiver operating characteristic curve in the derivation dataset for 
1-year long-term disability claim closure in disability claimants. 
Note: Area under the curve estimate is 0.70. 
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Figure 6. Expected versus predicted probability of long-term disability claim clo- 
sure based on deciles of risk is depicted in the validation dataset. The corre- 
sponding Hosmer-Lemeshow test was 10.7 with a p-value of 0.1. 
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Figure 7. Receiver operating characteristic curve in the validation dataset for 
1-year long-term disability claim closure in disability claimants. 
Note: Are a under the curve estimate is 0.69. 


a close relation between predicted and observed probability of 
claim closure (Figure 3). 


Strengths and limitations 
This is a large prognostic model study of disability claimants 


with depression with an adequate data source and analysis 
plan [16]. We defined objectives, prognostic markers and out- 
comes as ad-hoc. Explicit eligibility criteria limited the risk for 
misclassification. The prognostic markers incorporated in our 
model have objective and reproducible measurement properties 
[16]. Other strengths of our study included a priori creation of 
regression models and the anticipated direction of effects of 
included prognostic markers (Table 3). 

We were limited by the existing database for quality and quan- 
tity of the variables. Another limitation is the absence of infor- 
mation about health risk behaviours, such as alcohol intake and 
drug dependence. As a result, we may have missed potentially 
important prognostic markers. Since we used an administrative 
database, diagnostic validity is expected to be low [17]. 


Findings in the context with previous evidence 

Depression is a common cause of disability. Improved under- 
standing of the factors associated with claim resolution may 
lead to strategies to improve prognosis among patients at high 
risk of prolonged claim duration [18-20]. A prior Canadian study 
indicated, unexpectedly, that residents of Quebec were more 
likely to resolve their LTD claim faster when compared to the 
Ontario residents (HR=1.93, 99% Cl=1.78-2.09) [7]. This is con- 
gruent with our findings. 

The association of province of residence with delayed recovery 
may be explained by differences in the quality and quantity of 
mental health services provided in Ontario and Quebec, or dif- 
ferences in claim management policies and processes in Que- 
bec. These findings require further exploration. 

Our findings are consistent with previous evidence that sug- 
gests a link between older age, female gender, and higher pre- 
disability salary with prolonged duration of disability claims 
[21, 22]. In contrast, our results do not support previous findings 
that have reported an association between low physical job de- 
mands and longer disability benefit duration [23]. 


Implications of our findings 

Further research is needed to explore the determinants of the 
differences in LTD claim closures between provinces. A pro- 
spective well-designed and well-conducted cohort study would 
provide more reliable evidence than our results. Exploration of 
the factors associated with disability claim duration will require 
linking records from several databases to retrieve complete in- 
formation about health care utilization. In the presence of im- 
portant prognostic markers, model performance will be likely to 
improve. Our model’s discrimination and calibration was suff- 
cient to warrant application in another database for validation. 
To conclude, we developed a prognostic model for predicting 
the possibility of LTD claim closure in Canadian workers dis- 
abled by depression. Our study found that disabled workers 
who reside in Ontario receive LTD benefits longer than similar 
patients who reside in Quebec. Further study is needed to de- 
termine if this association is affected by differences in patients, 
clinical management, or case management between provinces. 
Our prognostic model has modest but acceptable discrimina- 
tion and good calibration and warrants validation in an inde- 
pendent data set. 
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